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ABSTRACT 

This report provides historical background on the 
origins, development and procedures, of Educational Testing Service's 
(ETS's) Board of Review. Established in 1969, the Board of Review 
makes final decisions for all test scores of questionable validity. 
ETS cancels or withholds scores believed to be invalid. Reasons for 
invalid scores range from improper testing conditions to overt 
candidate misconduct. Between these extremes lie the problems of 
questionable validity. Cheating on tests has been a recognized 
problem since the College Entrance Examination Board's founding in 
1901. After the 1947 establishment of ETS, first test program 
directors, then a security officer (1956) were responsible for test 
score investigations. In the 1960 's the Law School Admission 
Council's concern led to a review of ETS test security procedures 
directed by Robert Smith, and the subsequent establishment of the 
Board of Review. While there have been ongoing policy changes and 
procedural refinements (most notably increasing reliance on 
sophisticated statistical methods and computer technology) the key 
element is unchanged. ETS interest is in the validity of the scores 
it reports, not in providing evidence or judgments of candidate 
misconduct. Court cases concerning score reporting have upheld ETS 
policies. (BS) 
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Treatttent of Scores of Questionable Validity: 
The Origins and Develoi»ient of the ETS Board of Review 

Educational Testing Service is obligated to test takers, score recipients » 
and test sponsors to report scores which reasonably quantify the abilities of 
of examinees. Concomitant ly, ETS cancels or withholds scores; it believes to 
be invalid and^ no less if not sore significant ly» establishes safeguards 
designed to prevent erroneous cancellations of valid scores. These obligations 
derive from ETS^s status as a not*^for-prof it educational organisation created 
to serve the public interest; from the ethical standards of the professional 
organisations to which ETS and its staff members belong; from the contracts 
under which ETS performs its services; and from the implied agreements with 
individual candidates to report valid scores in return for their registration 
fees . 

Invalid scores may result from defective test materials, mistimings, or 
other irregularities for which a test taker cannot be deemed responsible. 
Invalidity may also result from deliberate examinee misconduct. ETS has con-* 
sistently authorised its test supervisors to dismiss candidates caught in the 
act of using prohibited reference materials, giving or receiving assistance, 
or other prohibited behavior* Similarly, ETS may cancel scores after a test 
administration if it receives convinci: g evidence of misconduct* 

Between these two extremes of improper testing conditions and overt 
candidate misconduct lies the difficult problem of scores of questionable 
validity* The validity of a score may be questionable for reasons such as the 
following: a report that misconduct may have occurred; an improbable score 
gain o^^er a previous test; an unusual agreement with responses on another 
test paper; striking differences in handwriting among test materials 
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purportedly written by the same candidate; or an indication that a test taker 
may have had access to test questions before the examination (preknowledge) • 

In 1969 » the ETS Trustees adopted a new policy governing treatment of 
questionably valid scores. Through the establishment of the Board of Review, 
it substantially strengthened the organisation's assurance that decisions 
concerning the validity of test scores would be made with the utmost care; 
that all test takers whose scores %ns»re investigated would be treated fairly; 
and that such score cancellations would be motivated solely by a concern for 
score validity and not by an intent to penalize candidates who may have 
violated rules of test administration. To put the Board's origins in perspec^ 
tive^ it may be helpful to review earlier test security practices. 

By the time the College Entrance Examination Board was founded in 1900, 
cheating on tests was a recognized phenomenon of human behavior J That the 
College Board founders were aware that test takers might engage in misconduct 
is manifestly evidenced by the Board's Document No. 2 of February 1, 1901. 
The section entitled "Instructions to Candidates for Examination" warned the 
first College Board candidates that the presence of contraband material would 
be cause for dismissal from the examination room, as would be giving or 
receiving assistance. "Upon thin subject the judgment of the supervisor in 
charge of the examination will be final and without appeal." Document No. 2 
does not mention the possibility that an irregularity might be discovered 
after the test administiat ion; anticipated or not » such troublesome events 
soon occurred. 

Some indication of the ext<?nt of cheating discovered on early College 
Board tests is provided by an archival gem: a stenographer's transcript of a 
conference of supervisors called together in 1926 to discuss the forthcoming 



first administration of the Scholastic Aptitude Test. Comientiiig on the 



candidate misconduct problem, the Board's Secretary » Thomas Fiske, stated, 
do not believe that the half dosen cases that ne discover [every year] are an 

insignificant proportion of the real nu^er of cases that exist. "2 A few 

# 

minutes later, Fiske commented: 

In Boston and Cambridge... there have been... a good 
many cases of impersonation and cheating* One year... 
t%ro boys were expelled frca the Boston Latin School... 
and their parents raised a terrible row and said, **Why, 
you are punishing our boys, ruining their careers, and 
all they are guilty of is the thing on the basis of which 
* Mayor Curley was elected Mayor of Boston*" Curley was in 
jail for impersonating other people at Civil Service 
examinations, and when his political enemies attackc* him, 
he patted himself on the chest and said, **Why, I think 
a fellow who would go to jail to help another fellow get 
a job mist be a pretty good sort of fellow," and the 
people of Boston agreed....^ 

While some attempted impersonations, other test takers brought crib 
sheets into the testing room. Fiske seemed amused that every year readers 
found reference materials i.i the answer books of forgetful cam i ates. In 
1923, one student explained that, at the suggestion of his teacher, he had 
removed pages from his gec»etry textbook in order to study them on the 
streetcar he took to the examination center. During the test, when he removed 
his handkerchief from his pocket, the pages, he assumed, must have fallen 
unnoticed into his answer book.^ 

Fiske described the Board's policies and procedures for dealing with 
these occasional cases as follows: 

In cases of suspected cheating, we always try to get 
the advice of the supervisor and the head of the 
preparatory school from i#hich the candidate comes, 
vnienever we suspect a candidate of having been guilty 
of a violation of the rules, we always investigate as 
fully as possible and consult everybody who could 
possibly shed light on the subject.... 



We have a rogue's gallery [in the office], but it does 
not really aean that the boy is a rogue. It means he 
is suspected of an irregularity or a violation of our 
rules. We have to refrain fro« actually accusing 
candidates of dishonesty....^ 

Because most College Board candidates of that era were preparatory school 

students well fcno%m to the test supervisors, ispersonat ions must have been 

quite unusual « But Fiske probably underestimated the extent of other forma of 

cheating. For every candidate vho left a crib sheet in his answer book, there 

must have been many others sufficiently composed to cake their textbook 

excisions hooie. 

Fiske*s willingness to discuss investigations in progress with school 
officials differs strikingly from current practice, which reflects a greater 
concern for test taker privacy* More consistent with current policy is 
Fiske *s distinction between dishonesty and rule breaking. As will be seen 
below, ETS has continued to avoid character i2ing candidate misbehavior in 
legal terminology* 

It seems curious today that Fiske did not specify copying from other 
papers as a cheating method. At the time, Fiske had little experience with 
the "new^type" multiple-choice tests; in the absence of effective deterrents, 
&hort answer tests facilitate illicit transcription. The multiple-choice test 
format created a potential test security problem soon recognised by specialists. 

In 19^ ^ 8 year after the supervisors'meet ing, Charles Bird of the 
University of Minnesota published his article, **The Detection of Cheating in 
Objective Examinations," perhaps the earliest explanation in print of the 
basic rationale still used for making determinations in copying cases: *'We 
can tell whether the identical wrong answers in two papers exceed a nij«ber 
which is possible by chance.'* Although his statistical procedures were crude 



by modern standards, current methods share Bird's asstmption that '*a student 
who secures infonsation surreptitiously from another paper is seldcm capable 
of discriminating right (rem %rrong answers in an objective examination."^ 

Itespite the availability of this technique, it does not appear that the 
College Board imediately felt in need of it. After the introduction of the 
Scholastic Aptitude Test, the Board's written procedures for dealing with 
cheating continued to be limited to admonishing candidates to behave themselves 
and encouraging invigilators to be on the qui vive. Remarkably, in the first 
decade of SAT administrations, not one candidate's score was cancelled for 
misconduct. In fact, according to Cecil BrQlyer, a central figure in SAT 
affairs from 1927 to 1936, there were not even any invest igations*^ 

For a number of years after ETS was established in 1947, testing program 
directors were responsible, with the assistance of test administration staff, 
for investigating questioned scores and determining whether to invalidate 
them. What evidence there is suggests that investigations were quite unusual. 

In 1955, a Task Force on Physical Security of Tests, consisting of 
William Bretnall, Catherine Sharp, Ned Terral, and William Van Cleve, made 
an in-depth survey of test security at ETS and recoimended that a Security 
Officer be appointed. ^ ^i^. y^n Cleve, ^o worked in Supervisor Relations, 
was named to the new position in 1956. Although the task force report had not 
mentioned impersonations or communication (its concern was to minimise test 
book losses), he was given responsibility for handling all test security 
problems. Daring his first year, there was a difficult impersonation case in 
which Bretnall's father, a private detective, lent a hand. As his son recalled 

The case began in August of 1956 and dragged on through 
the end of January 1957. In between were continuing 
threats of a law suit, appeals to Governor Ribicoff of 
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Connecticut, accusations that ve were persecuting 
Armenians in general, interviews by «y father with 25 or 
30 people, consultations with lawyers, and an endless 
streffiB of letters^ memoranda and phone calls. Finally, 
six months after it all began, the candidate and his 
impersonator admitted their deed. It was a great relief 
to all concerned. 9 

Such experiences indicated that ETS needed a trained investigator as 
Security Officer. Perhaps this change, made in 1958. was responsible for 
a dramatic increase in the number of confirmed impersonations: from nine 
in 1957-1958 to fifty-nine in 1958-1959. In the latter year, another five 
scores were cancelled for cheating by other means. 

In 1966, the ETS Security Officer, Captain Paul D. Williams, a former 
Navy officer with intelligence experience, described the procedures followed 
by the Test Security Office. Williams stated that after gathering the avail- 
able evidence, including an opinion from a handwriting expert if impersonation 
was suspected, he would contact the candidate, usually through a school official, 
(Until about 1960, the interview was almost always conducted in person, but 
by 1964-1965, the case load had increased to the point that 95 percent of the 
discussions were held over the telephone.) Then, as Williams continued: 

The candidate is apprized of the investigator's identity, 
the reason for the investigation, a resume of the evidence 
indicating that the candidate has disqualified himself; 
namely the score comparison, the indication of impersona- 
tion, or of copying, as the case may b«.>, and any other 
pertinent observations the investigator has made. The 
candidate is then requested to c<^ent and if he confirms 
the purport of the evidence, a statement affirming th% 
irregularity and agreeing to the invalidation of the 
questioned scores is dictated to the candidate. He is 
requested to forward this statement in writing to the 
Security Office. . . . 

If he should refuse to confirm the evidence and indicate 
positively that he had not be<*n involved in any irregu- 
larity, the investigator will suggest to him that he 
retake the test at a special administration.... If 
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the candidate refuses the opporfMnity to be reteated^ 
the investigator vill infora him that a complete account 
of the investigation will be miKle to each of the 
inati tut ions i^ich have received his questioned scores* 

If the candidate vould not agree to cancel his or her score and refused to 
retest. ETS would send the score report to the score recipients with a "dubious 
validity letter'* which "would draw no conclusions/* although it would describe 
the evidence suggestive of misconduct. Use of the dubious validity letter was 
infrequent since most confronted examinees agreed to cancel their scores: 85 
percent in 1964-1965 • 

Some score recipients, notably law schools, wanted ETS to do more. In 
1964| the Law School Admission Council *s Executive Committee appointed an Ad 
Hoc Comnittee on Test Security to look into problems of cheating and to review 
ETS investigative practices* Indirect ly, this Committee contributed to the 
Coraation of the ETS Board of Review. 

As compared to test sponsors of other ETS programs in the 1960*8. the Law 
School Admission Council had already shown a significant interest in infrac- 
tions. For example, the LSAT was the only progrm which required Test Security 
to check the validily of increased repeater scores as a standard procedure; at 
this time a gain of 100 points^j: more triggered an investigation.^^ The 
law schools* stated reason for their particular concern was that they were 
required to report on the character of their graduates when they applied for 
bar examinations. 

The LSAC Ad Hoc Committee presented its findings to the Council in 1965« 
Their report sumaarised existing ETS procedures » including the use of compari^ 
sons of w ^ng answers in copying casea^ and termed them ** adequate.*' The 
Committee did make a nmber of recommendations for change » most of them minor , 
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but of particular interest was a profKisal that if a candidate persisted in 
denying misconduct » ETS should arrange **a hearing panel of three persons 
through the American Academy of Arbitrators or the local bar association** to 
make a final decision* 

In response to the Committee's report, ETS Counsel John Craham accepted 
some of the suggest ions » including sending '*an information copy to the testee 
of any final letters to score recipients about the validity of his score,'* 
But Grahm believed that some of the Committee's other suggestions irould make 
the investigations inadvisably rescmible criminal proceedings. '*ETS/' stated 
Graham 9 "has formulated its investigative procedures to provide the maximum 
protection to the integrity of reported test scores consistent with basic 
fairness to the teStee and minimisation of potential liability against ETS.., 
every effort is made to avoid the implication that the investigation is a 
criminal proceeding. For example, statements obtained from suspected testees 
are called admissions, not confessions." 

Graham ''strongly opposed" the Cosasiittee' s suggestion of a review board, 
not so much over the score validity vs. misconduct emphasis, but for %rhat he 
believed were practical reasons; the expense; the administrative burden and 
psychological effect on test security personnel whose actions would be 
subject to review; the abuse of the procedure, such as for a delaying tactic; 
and the likelihood that the board's decision would not be binding and provide 
no protection against I itigation. 

As a result of Graham's coiraents, the LSAC Executive Committee requested 
the Security Ccmaittee to reconsider its report and consult further with ETS. 
In 1966, the Security Comittee reported back with revised proposals that 
represented a compromise between its initial recommendations and Crah^'s 

10 
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objections. But the Cmnilttee stood first in its desire for a hearing board 
for disputed cases p as did Graham in his opposit ion. 

The lav school concerns led ETS Vice President Robert J. Solmon to ask 
Robert E. Staithp Executive Director of General Progrmsp to take a close look 
at the test security operation. Subsequent ly^ Smith and Solomon agreed that 
changes were needed. As Smith recalled p '*One thing that bothered ua in the 
very beginning vas that, in our viewp there nas no opportunity for the young*- 
ster really to get good advice" before responding to ETS*8 interrogator. *^We 
vere also concerned p** continued Sraithp "about the business of telling the test 
takers thaC the thing they needed to do" va^ to sign a dicta^ed confession. 
Nor did they like the dubious validity letter. Even though the existing proce- 
dures were based on a concern focusing on score validity rather than vrongdoingp 
they felt the result was overly punitive and p as Smith concluded p "We should 
refrain from any actions that made the business more public than was absolutely 
necessary ."^^ In order to develop viable al ternat ives p Solomon appointed a 
Committee to Review ETS Security ProcedureSp chaired by Smith, in November 



In his "Presentation on Cheating" to the ETS Board of Trustees on May 7p 
1968t Smith presented his ccnmittee's tw alternatives. Smith's preferred 
poseibtlity was to advise score recipients when a question arose concerning a 
candidate's score p before an investigation which would result in a determina-* 
tion of the score's validity. If the institution wishedp it could contact 
the candidate and ask for a retest . ETS would not have to cancel any scores. 

The Trustees strongly rejected this radical proposal p^^ stating that it 
was ETS's responsibility to determine if scores wre valid p and asked ETS to 
proceed along the lines of Smith's other alternative p which featured some of 
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Bob Smith, 1970 
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the p&ocedures introduced later the Board of Review was established: no 

request that candidates sign a statement of admission; offering all candidates 
an opportunity to re^est ; refunding fees to those whose scores were cancelled; 
and providing no explanation to score recipients of the reason for a cancella- 
tion. However 9 in this proposal, final determinations of score validity would 
be made jointly by the testing program director and the Security Officer. "A 
hearing board could be resorted to where unusual circumstances recoimnended 
it-" 

By August 5» 1^:* Smith and others at ETS had determined that it would 
be preferable to have an internal board, independent of both Test Security and 
Program Direction, make final decisions in all cases, leaving other features 
of this plan largely the same.^^ In December, the Trustees approved this plan, 
codified as the General Policy on Questioned Test Scores'* and the ETS officers 
appointed the Board of Review. Charter members were: William A. Angoff, 
Marion G, Epstein, John S. Kramer (ETS Counsel), and Robert E. Smith, Chairman. 
The first formal meeting of the Board was held on January 9, 1969. 

The policy states, that in order for ETS to fulfill its "obligation to 
deal fairly with the candidate as well ns to assure the authenticity of the 
scores to the recipients,** the available evidence concerning a questioned 
score is presented {by the Test Security Office) to a "Board of Review consist- 
ing of the ETS legal counsel and three senior professional staff members not 
directly responsible for the administration of the test programs concerned." 
Three members of the Board constitute a quorum. If therf* is unanimous agree- 
ment that the validity of a score is in doubt, a registered letter is sent 
to the candidate, stating that the Board will cancel the score unless the 
candidate can confirm by a retest or an adequate explanation* 
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At the heart of the policy is the fundmental principle that *'the proper 
interest of ETS rests vith the authenticity of the scores it reports and not 
vith providing evidence or a judgment cf misconduct by candidates • Punitive 
intent toward the candidates'* should be avoided. Accordingly » even if the 
Board of Reviev has strong evidence of test taker misconduct , it confirms a 
questioned score if the individual can demonstrate that the score is valid; 
in fact, it will not question a probable miscreant's score in the first place 
if available evidence suggests that retesting %fOuld confirm. If scores are 
cancelled, the candidate's fees are returned and the scores removed from ETS 
records. Score recipients are not provided ^he reason for the Board's cancel la- 
tion of the score. ^ 

Most specifics of the General Policy are still in effect. Differences 
concern the sise^ and composition of the Board (more members almost iimaediately 
and the exclusion of legal counsel in 1980) and the following supplemental 
features . 

The first major addition to the policy occurred in the Board's second 
year. For some time, Test ^curity had been using the computer to identify 
the LSAT candidates with score gains sufficiently large to warrant further 
investigation. At their Spring 1968 meeting, the ETS Trustees had suggested 
that ETS investigate the feasibility of expanding the application of this 
technique to other prograns. In February 1970, the Board of Review adopted 
its '^Recommended Supplementary Policy on Questioned Test ScoreSi" which 
mandated score gain checks by computer in other national programs. The 
Scholastic Aptitude Test was the first new program to which the technique was 
applied. The triggering cutting (difference) score for the SAT was established 
by research conducted by Mr. Angoff, who had found that above a certain score 



ERIC 




-12- 

gain on the December 1968 SAT, 88 percent of the answer sheets demonstrated 
clear evidence of impersonation or comunication. 

The Board noted in its new policy statement that a major advantage of 
computer-identified cases was that it permitted scores to be questioned before 
they were reported; the result was less embarrassment for those whose scores 
were subsequently cancelled. Another effect, of course , was to identify cases 
which might never have been discovered » Signif icantly, the Board iamiediately 
adopted a rule never to cancel a candidate's score on the sole basis of an 
improbable score gain; it directed Test Security not to present such cases to 
the Board if no other evidence suggestive of invalidity could be found ,20 

Despite the Board* s attempts to be fair to those whose scores were 
quest ionedi some crndidates and parents responded quite negatively to the 
Board's initial letter. To ameliorate this problem^ the Board tried a short-" 
lived ''pre- invest igat ion procedure as described in a policy statement Smith 
issued in February 1975. 

Under the new procedure, Admissions Testing Program and Graduate Record 
Examinations candidates whose scores were flagged due to large score differ- 
ences were sent a letter informing them that an investigation would soon be 
initiated. The letter also offered an opportunity to cancel or retest before 
an initial judgment was made concerning the validity of the score. The 
supposed advantage of this procedure for the candidates was to ''minimize any 
accusation of misconduct'' and for ETS^ to save investigative costs for those 
who authorized cancellation. Despite careful wording, sc^ie students perceived 
the letter as an accusation and felt that ETS should investigate further 
before contacting them; reaction was sufficiently negative that the procedure 
was dropped after a few years* 

o 15 
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In 1976 » as a result of a review of ETS test security practices by the 
College Board, questioned scores which had not been reported began to be 
placed in a "suspense*' file untili if ever, the test taker responded in 
writing to the Board's initial decision to question the score* In 19tt4| the 
Board of Review returned to its earlier policy of cancelling scores if the 
candidate does not respond, but with a longer waiting period when scores have 
not been reported. 

A nore lasting innovation officially introduced in 1978 is to offer the 
opportunity to have all of the evidence sent to the institution designated 
to receive the score, if the institution agrees. 22 

Another innovation, still current, is arbitration, proposed initially, it 
may be recalled, by the LSAC Ad Hoc Committee on Test Security in 1963. For 
LSAT candidates, the option of submitting the case to an arbitrator appointed 
by the American Arbitration Association was made available in 1973. In 1981, 
the College Board agreed to make arbitration an option in its testing programs 
and the procedure was adopted subsequently in the GRE and GMAT programs. 

In addition to these policy changes, there have been refinements to 

internal Board of Review procedures and, under its direction, those of the 

* 

Test Security Office. Perhaps most significant has been the development, 
application, and increased reliance on statistical indices used to provide a 
probability estimate that the correspondence of incorrect responses in a pair 
of answer sheets could have occurred by chance. The first such indices were 
developed by William Angoff in 1970 and revised in 1973 by Louis Lavine, who 
in 1976 became the second chairman of the Board. In 1979, Frederick Kling 
developed the currently used Index K. Recently, Index K has made feasible 
a serendipitous method of identifying cases to be brought to the Board for a 
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decision. These '^developed cases" are discovered occasionally when the 
«. MMputer is used to run cc»iparisons of all possible pairs of a group of answer 
sheets to detersiine the probable source » if any» for a test taker suspected of 
copying. (The all-pairs technique is also used when collusion or pre-knowledge 
of a test is suspected of an indeterminate nimber of candidates.) If strong 
K's are found. Test Security submits the developed cases to the Board. 

The use of increasingly sophisticated statistical methods and computer 
technology has had several effects on the Board's work* First » although some 
scores continue to be questioned by score recipients, the majority of suspected 
irregularities are discovered first at ETS and are resolved before scores are 
reported. Second , the computer has significantly increased the number of cases 
coming to the Board, despite additional measures implemented to prevent irreg- 
ulariti s at the test centers. This increase to about 2,(K)0 cases annually 
has necessitated a gradual expansion of Board membership to seventeen; the 
growth is in large measure a consequence of the computerized score difference 
check, although increased candidate volume and other less tangible factors may 
also be involved. Third, the score difference check and Index K may have 
figured in an increase of the proportion of cases involving probable (ionnnuni- 
cation to almost half, with the majority still impersonations. 

The nature of the changes to Board procedures suggest that although the 
basics of Board policy have remained unchanged^ there is a continuing effort 
by both ETS and test sponsors to review and refine Board practices. The 
courts have also provided evaluations of ETS test security policies and have ^\ 
found them satisfactory, even laudable. Although threats of lawsuits have 
been frequent, only a few cases have come to trial. In one early case before 
the establishment of the Board, involving probable preknowledge on the National 
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Teacher Examinations , the judge concluded, "The evidence is circumstantial » 
but circuastant ial evidence is not to be excluded. Circumstantial evidence is 
a basts for findings such more severe and much more harsh than those that would 
result in this case. To deny circumstantial evidence is to deny the ability 
of the human mind to reason ^"23 

Court cases after the Board's founding have also upheld ETS policies. In 
1983, there wa8 a considerable amount of public attention concerning a probable 
preknowledge case involving four SAT candidates. A key issue was whether ETS 
had to prove that cheating occurred in order to have a reasonable basis to 
question the candidates* scores. Finding ftfr ETS, the judge concluded that 
"ETS may not properly be forced to inquire into questioned scores in the manner 
of a law enforcement agency or to adjudicate guilt or innocence in the manner 
of a court. "24 jhis decision, upheld on appeal, further confirmed the wisdom 
of the key element in ETS' test security policies, developed in the 1930 's and 
enhanced by the establishment of the Board of Review in 1969, that ETS appro- 
priate concern in this area should be score validity. The Board continues to 
function on this principle. 

Gary D. Saretzky 
September 7, 1984 
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Footnotes 



Misconduct on examinations is undoubtedly as old as testing itself but 
its emergence as a potentially large**scale problem probably occurred %rhen stand- 
ardized tests vere introduced in China as a requirement for government service. 
Particularly in their mature form during the Ming to Qing djniasties (1368**1911) t 
"the examinations were in fact a cornerstone of the social and political 
edifice. and the centrepiece of the life and lore of the scholar-gentry-* 
official class" to which success on the tests provic|ed entry* Recognising that 
the temptation to cheat was very greats the authorities instituted extraordinary 
preventive measures. For the provincial and metrofKilitan examinations , test 
administrations were conducted in huge walled cc^pounda resembling prisons, 
tightly supervised by armed guards and ex4niner8* CandHates were stripped 
and searched before admittance » then locked up in virtually bare cells for up 
to four days of testing. Tbere^ each morning, they were brought their food 
and examination questions by deaf mutes, who w>uld pick up the papers at the 
end of the day. 

Despite these stringent prophylactic policies, somfe examinees attempted 
to achieve higher scores than they deserved. Occasionally, test develop&ent 
officials (and their unlucky subordinates) were beheaded for accepting money 
to reveal questions in advance. Another technique was to bribe the deaf mute 
either to allow reference materials to be brought in (baked in cakes, for 
example) or to take questions out to be completed by a confederate in an 
adjoining cell and returned • Impersonation was also practiced, as were 
underhanded methods to influence the grading process. (Bernard Luk, "The 
Civil Service Examinations in Late Imperial China," Orientations , 13 , 21.) 

"Conference of Supervisors Held in the Trustees' Room, Columbia Library, 
Saturday, January 9, 1926, at 10:30 a.m.," Benjamin Gotthelf, Shorthand Report^ 
ing, 154 Nassau Street, New York City, 93. 

^Ibid , , 103-104. 

Sbid. , 92. 

^Ibid . , 94, 98. 

^Charles Bird, "The Detection of Cheating in Objective Examinations," 
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